{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第4阶段_第6讲_综合分析实战\n",
    "\n",
    "## 学习目标\n",
    "1. 掌握完整的数据分析项目流程\n",
    "2. 综合运用数据清洗、EDA、统计分析、建模的方法\n",
    "3. 学会从业务问题出发,选择合适的分析方法\n",
    "4. 能够撰写专业的数据分析报告\n",
    "5. 掌握数据可视化和洞察提炼技巧\n",
    "6. 完成真实业务场景的端到端分析\n",
    "\n",
    "## 综合项目:电商平台用户运营分析\n",
    "\n",
    "### 项目背景\n",
    "\n",
    "某电商平台运营部门希望通过数据分析:\n",
    "1. 了解用户的购买行为特征\n",
    "2. 识别高价值用户群体\n",
    "3. 分析影响用户消费的关键因素\n",
    "4. 预测用户未来消费金额\n",
    "5. 为营销活动提供数据支持\n",
    "\n",
    "### 数据说明\n",
    "\n",
    "包含2000名用户的行为数据:\n",
    "- **用户属性**: 年龄、性别、会员等级、注册时长\n",
    "- **行为数据**: 登录次数、浏览时长、购买次数、消费金额\n",
    "- **商品偏好**: 主要购买类目、平均客单价\n",
    "- **满意度**: 客户评分、投诉次数、复购率\n",
    "\n",
    "### 分析流程\n",
    "\n",
    "```\n",
    "业务理解 → 数据准备 → EDA探索 → 用户分群 → 影响因素分析 → 预测建模 → 洞察总结 → 报告撰写\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入所有必要的库\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "# 统计和建模\n",
    "from scipy import stats\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import mean_squared_error, r2_score, mean_absolute_error\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.cluster import KMeans\n",
    "import statsmodels.api as sm\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "# 设置中文显示\n",
    "plt.rcParams['font.sans-serif'] = ['Arial Unicode MS', 'SimHei']\n",
    "plt.rcParams['axes.unicode_minus'] = False\n",
    "\n",
    "# 设置样式\n",
    "sns.set_style('whitegrid')\n",
    "sns.set_palette('husl')\n",
    "\n",
    "# 设置显示选项\n",
    "pd.set_option('display.max_columns', None)\n",
    "pd.set_option('display.width', 1000)\n",
    "pd.set_option('display.float_format', '{:.2f}'.format)\n",
    "\n",
    "# 设置随机种子\n",
    "np.random.seed(42)\n",
    "\n",
    "print(\"=\"*100)\n",
    "print(\"🎯 电商平台用户运营分析项目\")\n",
    "print(\"=\"*100)\n",
    "print(\"\\n✅ 环境配置完成!\")\n",
    "print(f\"Pandas版本: {pd.__version__}\")\n",
    "print(f\"NumPy版本: {np.__version__}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第一阶段:业务理解与数据准备\n",
    "\n",
    "### 1.1 明确业务问题"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义业务问题\n",
    "business_questions = {\n",
    "    '问题1': '用户的购买行为有什么特征?不同用户群体的差异在哪里?',\n",
    "    '问题2': '哪些因素对用户消费金额影响最大?',\n",
    "    '问题3': '如何识别高价值用户?如何进行用户分群?',\n",
    "    '问题4': '能否预测用户未来的消费金额?',\n",
    "    '问题5': '针对不同用户群体应该采取什么运营策略?'\n",
    "}\n",
    "\n",
    "print(\"=\"*100)\n",
    "print(\"📋 核心业务问题\")\n",
    "print(\"=\"*100)\n",
    "for key, question in business_questions.items():\n",
    "    print(f\"\\n{key}: {question}\")\n",
    "print(\"\\n\" + \"=\"*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 生成模拟数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 生成电商用户数据\n",
    "n_users = 2000\n",
    "\n",
    "# 基础用户属性\n",
    "age = np.random.normal(32, 10, n_users).clip(18, 65).astype(int)\n",
    "gender = np.random.choice(['男', '女'], n_users, p=[0.48, 0.52])\n",
    "member_level = np.random.choice(['普通', '白银', '黄金', '铂金', '钻石'], n_users, p=[0.35, 0.30, 0.20, 0.10, 0.05])\n",
    "register_days = np.random.poisson(lam=365, size=n_users).clip(30, 1500)\n",
    "city_tier = np.random.choice(['一线', '二线', '三线', '四线'], n_users, p=[0.25, 0.35, 0.25, 0.15])\n",
    "\n",
    "# 行为数据\n",
    "login_count = np.random.poisson(lam=20, size=n_users).clip(1, 150)\n",
    "browse_time = np.random.exponential(scale=30, size=n_users).clip(5, 200)  # 分钟\n",
    "purchase_count = np.random.poisson(lam=8, size=n_users).clip(1, 100)\n",
    "monthly_spending = np.random.gamma(shape=2, scale=800, size=n_users).clip(100, 20000)\n",
    "\n",
    "# 商品偏好\n",
    "main_category = np.random.choice(['电子产品', '服装', '食品', '家居', '图书'], n_users, p=[0.25, 0.30, 0.20, 0.15, 0.10])\n",
    "\n",
    "# 满意度指标\n",
    "rating = np.random.choice([1, 2, 3, 4, 5], n_users, p=[0.02, 0.05, 0.15, 0.45, 0.33])\n",
    "complaint_count = np.random.choice([0, 1, 2, 3], n_users, p=[0.70, 0.20, 0.08, 0.02])\n",
    "repurchase = np.random.choice(['是', '否'], n_users, p=[0.68, 0.32])\n",
    "\n",
    "# 创建DataFrame\n",
    "ecommerce_data = pd.DataFrame({\n",
    "    '用户ID': [f'U{str(i).zfill(5)}' for i in range(1, n_users+1)],\n",
    "    '年龄': age,\n",
    "    '性别': gender,\n",
    "    '会员等级': member_level,\n",
    "    '注册天数': register_days,\n",
    "    '城市等级': city_tier,\n",
    "    '月登录次数': login_count,\n",
    "    '月浏览时长': browse_time,\n",
    "    '月购买次数': purchase_count,\n",
    "    '月消费金额': monthly_spending,\n",
    "    '主要类目': main_category,\n",
    "    '客户评分': rating,\n",
    "    '投诉次数': complaint_count,\n",
    "    '是否复购': repurchase\n",
    "})\n",
    "\n",
    "# 根据会员等级调整消费金额(更真实)\n",
    "level_multiplier = {'普通': 0.7, '白银': 1.0, '黄金': 1.5, '铂金': 2.2, '钻石': 3.5}\n",
    "ecommerce_data['月消费金额'] = ecommerce_data.apply(\n",
    "    lambda row: row['月消费金额'] * level_multiplier[row['会员等级']], axis=1\n",
    ")\n",
    "\n",
    "# 根据年龄调整行为(年轻人更活跃)\n",
    "ecommerce_data.loc[ecommerce_data['年龄'] < 30, '月登录次数'] = \\\n",
    "    (ecommerce_data.loc[ecommerce_data['年龄'] < 30, '月登录次数'] * 1.3).astype(int)\n",
    "\n",
    "# 根据注册天数调整行为(老用户更稳定)\n",
    "ecommerce_data.loc[ecommerce_data['注册天数'] > 730, '月购买次数'] = \\\n",
    "    (ecommerce_data.loc[ecommerce_data['注册天数'] > 730, '月购买次数'] * 1.2).astype(int)\n",
    "\n",
    "# 计算客单价\n",
    "ecommerce_data['客单价'] = ecommerce_data['月消费金额'] / ecommerce_data['月购买次数']\n",
    "\n",
    "# 添加一些缺失值(真实场景)\n",
    "missing_indices = np.random.choice(ecommerce_data.index, 50, replace=False)\n",
    "ecommerce_data.loc[missing_indices, '月浏览时长'] = np.nan\n",
    "missing_indices2 = np.random.choice(ecommerce_data.index, 30, replace=False)\n",
    "ecommerce_data.loc[missing_indices2, '客户评分'] = np.nan\n",
    "\n",
    "# 添加一些异常值\n",
    "outlier_indices = np.random.choice(ecommerce_data.index, 10, replace=False)\n",
    "ecommerce_data.loc[outlier_indices, '月消费金额'] = np.random.uniform(30000, 50000, 10)\n",
    "\n",
    "# 保留小数\n",
    "ecommerce_data['月浏览时长'] = ecommerce_data['月浏览时长'].round(1)\n",
    "ecommerce_data['月消费金额'] = ecommerce_data['月消费金额'].round(2)\n",
    "ecommerce_data['客单价'] = ecommerce_data['客单价'].round(2)\n",
    "\n",
    "print(\"\\n✅ 数据集生成完成!\")\n",
    "print(f\"\\n数据集规模: {ecommerce_data.shape[0]}行 × {ecommerce_data.shape[1]}列\")\n",
    "print(\"\\n前10条数据预览:\")\n",
    "print(ecommerce_data.head(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 数据质量检查"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"=\"*100)\n",
    "print(\"🔍 数据质量检查\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 1. 基本信息\n",
    "print(\"\\n【1. 数据基本信息】\")\n",
    "print(\"-\"*100)\n",
    "ecommerce_data.info()\n",
    "\n",
    "# 2. 缺失值统计\n",
    "print(\"\\n【2. 缺失值统计】\")\n",
    "print(\"-\"*100)\n",
    "missing_stats = pd.DataFrame({\n",
    "    '缺失数量': ecommerce_data.isnull().sum(),\n",
    "    '缺失比例(%)': (ecommerce_data.isnull().sum() / len(ecommerce_data) * 100).round(2)\n",
    "})\n",
    "missing_stats = missing_stats[missing_stats['缺失数量'] > 0].sort_values('缺失数量', ascending=False)\n",
    "if len(missing_stats) > 0:\n",
    "    print(missing_stats)\n",
    "    print(f\"\\n总缺失值: {ecommerce_data.isnull().sum().sum()}个\")\n",
    "else:\n",
    "    print(\"✅ 无缺失值!\")\n",
    "\n",
    "# 3. 重复值检查\n",
    "print(\"\\n【3. 重复值检查】\")\n",
    "print(\"-\"*100)\n",
    "duplicate_count = ecommerce_data.duplicated().sum()\n",
    "print(f\"重复行数: {duplicate_count}\")\n",
    "if duplicate_count == 0:\n",
    "    print(\"✅ 无重复数据!\")\n",
    "\n",
    "# 4. 数据类型\n",
    "print(\"\\n【4. 数据类型分布】\")\n",
    "print(\"-\"*100)\n",
    "print(f\"数值型字段: {len(ecommerce_data.select_dtypes(include=[np.number]).columns)}个\")\n",
    "print(f\"对象型字段: {len(ecommerce_data.select_dtypes(include=['object']).columns)}个\")\n",
    "\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"✅ 数据质量检查完成!\")\n",
    "print(\"=\"*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.4 数据清洗"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"🧹 数据清洗\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 复制数据进行清洗\n",
    "df = ecommerce_data.copy()\n",
    "\n",
    "# 1. 处理缺失值\n",
    "print(\"\\n【1. 处理缺失值】\")\n",
    "print(\"-\"*100)\n",
    "print(\"策略:\")\n",
    "print(\"  - 月浏览时长: 用中位数填充(缺失较少,用中位数更稳健)\")\n",
    "print(\"  - 客户评分: 用众数填充(评分是离散值)\")\n",
    "\n",
    "before_missing = df.isnull().sum().sum()\n",
    "df['月浏览时长'].fillna(df['月浏览时长'].median(), inplace=True)\n",
    "df['客户评分'].fillna(df['客户评分'].mode()[0], inplace=True)\n",
    "after_missing = df.isnull().sum().sum()\n",
    "\n",
    "print(f\"\\n清洗前缺失值: {before_missing}个\")\n",
    "print(f\"清洗后缺失值: {after_missing}个\")\n",
    "print(\"✅ 缺失值处理完成!\")\n",
    "\n",
    "# 2. 异常值检测\n",
    "print(\"\\n【2. 异常值检测(IQR方法)】\")\n",
    "print(\"-\"*100)\n",
    "\n",
    "def detect_outliers_iqr(data, column):\n",
    "    Q1 = data[column].quantile(0.25)\n",
    "    Q3 = data[column].quantile(0.75)\n",
    "    IQR = Q3 - Q1\n",
    "    lower_bound = Q1 - 1.5 * IQR\n",
    "    upper_bound = Q3 + 1.5 * IQR\n",
    "    outliers = data[(data[column] < lower_bound) | (data[column] > upper_bound)]\n",
    "    return outliers, lower_bound, upper_bound\n",
    "\n",
    "# 检测月消费金额的异常值\n",
    "outliers, lower, upper = detect_outliers_iqr(df, '月消费金额')\n",
    "print(f\"月消费金额异常值检测:\")\n",
    "print(f\"  正常范围: [{lower:.2f}, {upper:.2f}]\")\n",
    "print(f\"  异常值数量: {len(outliers)}个 ({len(outliers)/len(df)*100:.2f}%)\")\n",
    "print(f\"  策略: 保留异常值(可能是真实的高消费用户),但在分析时需注意\")\n",
    "\n",
    "# 标记异常值\n",
    "df['是否异常消费'] = 0\n",
    "df.loc[(df['月消费金额'] < lower) | (df['月消费金额'] > upper), '是否异常消费'] = 1\n",
    "\n",
    "print(\"\\n✅ 异常值检测完成!\")\n",
    "\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(f\"✅ 数据清洗完成! 最终数据集: {df.shape[0]}行 × {df.shape[1]}列\")\n",
    "print(\"=\"*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第二阶段:探索性数据分析(EDA)\n",
    "\n",
    "### 2.1 整体数据分布"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"=\"*100)\n",
    "print(\"📊 数据分布概览\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 数值型变量统计\n",
    "numeric_cols = ['年龄', '注册天数', '月登录次数', '月浏览时长', '月购买次数', '月消费金额', '客单价']\n",
    "print(\"\\n【数值型变量统计摘要】\")\n",
    "print(\"-\"*100)\n",
    "desc_stats = df[numeric_cols].describe().T\n",
    "desc_stats['偏度'] = df[numeric_cols].skew()\n",
    "desc_stats['峰度'] = df[numeric_cols].kurt()\n",
    "print(desc_stats.round(2))\n",
    "\n",
    "# 分类变量统计\n",
    "print(\"\\n【分类变量分布】\")\n",
    "print(\"-\"*100)\n",
    "categorical_cols = ['性别', '会员等级', '城市等级', '主要类目', '是否复购']\n",
    "for col in categorical_cols:\n",
    "    print(f\"\\n{col}:\")\n",
    "    value_counts = df[col].value_counts()\n",
    "    for val, count in value_counts.items():\n",
    "        print(f\"  {val}: {count}个 ({count/len(df)*100:.1f}%)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 可视化:整体分布\n",
    "fig, axes = plt.subplots(3, 3, figsize=(18, 14))\n",
    "axes = axes.flatten()\n",
    "\n",
    "# 1-7: 数值型变量直方图\n",
    "for idx, col in enumerate(numeric_cols):\n",
    "    axes[idx].hist(df[col].dropna(), bins=40, color='steelblue', alpha=0.7, edgecolor='black')\n",
    "    axes[idx].axvline(df[col].mean(), color='red', linestyle='--', linewidth=2, label=f'均值={df[col].mean():.1f}')\n",
    "    axes[idx].axvline(df[col].median(), color='green', linestyle='--', linewidth=2, label=f'中位数={df[col].median():.1f}')\n",
    "    axes[idx].set_title(f'{col}分布', fontsize=11, fontweight='bold')\n",
    "    axes[idx].set_xlabel(col, fontsize=9)\n",
    "    axes[idx].set_ylabel('频数', fontsize=9)\n",
    "    axes[idx].legend(fontsize=8)\n",
    "    axes[idx].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 8: 会员等级分布\n",
    "level_order = ['普通', '白银', '黄金', '铂金', '钻石']\n",
    "level_counts = df['会员等级'].value_counts().reindex(level_order)\n",
    "colors = ['gray', 'silver', 'gold', 'lightblue', 'pink']\n",
    "axes[7].bar(level_counts.index, level_counts.values, color=colors, alpha=0.8, edgecolor='black')\n",
    "axes[7].set_title('会员等级分布', fontsize=11, fontweight='bold')\n",
    "axes[7].set_ylabel('用户数', fontsize=9)\n",
    "axes[7].tick_params(axis='x', rotation=45, labelsize=8)\n",
    "axes[7].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 9: 城市等级饼图\n",
    "city_counts = df['城市等级'].value_counts().reindex(['一线', '二线', '三线', '四线'])\n",
    "axes[8].pie(city_counts.values, labels=city_counts.index, autopct='%1.1f%%', \n",
    "            colors=['red', 'orange', 'yellow', 'green'], startangle=90)\n",
    "axes[8].set_title('城市等级分布', fontsize=11, fontweight='bold')\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n📊 关键发现:\")\n",
    "print(f\"  1. 用户年龄集中在{df['年龄'].quantile(0.25):.0f}-{df['年龄'].quantile(0.75):.0f}岁,呈正态分布\")\n",
    "print(f\"  2. 月消费金额右偏(偏度={df['月消费金额'].skew():.2f}),存在高消费用户\")\n",
    "print(f\"  3. 普通会员占比最高({level_counts['普通']/len(df)*100:.1f}%),钻石会员最少({level_counts['钻石']/len(df)*100:.1f}%)\")\n",
    "print(f\"  4. 复购率为{len(df[df['是否复购']=='是'])/len(df)*100:.1f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 相关性分析"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 计算相关系数矩阵\n",
    "corr_matrix = df[numeric_cols].corr()\n",
    "\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"🔗 变量相关性分析\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 找出与月消费金额高度相关的变量\n",
    "print(\"\\n【与月消费金额的相关系数】\")\n",
    "print(\"-\"*100)\n",
    "spending_corr = corr_matrix['月消费金额'].sort_values(ascending=False)\n",
    "for var, corr in spending_corr.items():\n",
    "    if var != '月消费金额':\n",
    "        if abs(corr) >= 0.5:\n",
    "            strength = \"强相关\"\n",
    "        elif abs(corr) >= 0.3:\n",
    "            strength = \"中等相关\"\n",
    "        else:\n",
    "            strength = \"弱相关\"\n",
    "        direction = \"正\" if corr > 0 else \"负\"\n",
    "        print(f\"  {var:<15}: {corr:>7.4f} ({strength},{direction})\")\n",
    "\n",
    "# 可视化相关系数矩阵\n",
    "fig, axes = plt.subplots(1, 2, figsize=(16, 6))\n",
    "\n",
    "# 左图:热力图\n",
    "sns.heatmap(corr_matrix, annot=True, fmt='.2f', cmap='coolwarm', center=0,\n",
    "            square=True, linewidths=1, cbar_kws={'label': '相关系数'},\n",
    "            vmin=-1, vmax=1, ax=axes[0])\n",
    "axes[0].set_title('变量相关系数热力图', fontsize=13, fontweight='bold')\n",
    "\n",
    "# 右图:月消费金额的相关系数柱状图\n",
    "spending_corr_plot = spending_corr.drop('月消费金额').sort_values()\n",
    "colors = ['red' if x < 0 else 'green' for x in spending_corr_plot.values]\n",
    "axes[1].barh(spending_corr_plot.index, spending_corr_plot.values, color=colors, alpha=0.7, edgecolor='black')\n",
    "axes[1].axvline(x=0, color='black', linestyle='-', linewidth=1)\n",
    "axes[1].set_xlabel('相关系数', fontsize=11)\n",
    "axes[1].set_title('各变量与月消费金额的相关性', fontsize=13, fontweight='bold')\n",
    "axes[1].grid(axis='x', alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n💡 相关性洞察:\")\n",
    "top_corr = spending_corr.drop('月消费金额').abs().sort_values(ascending=False).head(3)\n",
    "print(\"  与月消费金额相关性最强的3个因素:\")\n",
    "for i, (var, corr) in enumerate(top_corr.items(), 1):\n",
    "    print(f\"    {i}. {var} (|r|={corr:.4f})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 不同维度的对比分析"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 按会员等级分组分析\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"📊 按会员等级分组分析\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "level_stats = df.groupby('会员等级').agg({\n",
    "    '月消费金额': ['count', 'mean', 'median', 'std'],\n",
    "    '月购买次数': 'mean',\n",
    "    '月登录次数': 'mean',\n",
    "    '客单价': 'mean',\n",
    "    '客户评分': 'mean'\n",
    "}).round(2)\n",
    "\n",
    "level_stats.columns = ['_'.join(col).strip() for col in level_stats.columns.values]\n",
    "level_stats = level_stats.reindex(level_order)\n",
    "\n",
    "print(\"\\n【各会员等级关键指标】\")\n",
    "print(level_stats)\n",
    "\n",
    "# 可视化对比\n",
    "fig, axes = plt.subplots(2, 2, figsize=(15, 12))\n",
    "\n",
    "# 1. 各等级平均消费\n",
    "mean_spending = df.groupby('会员等级')['月消费金额'].mean().reindex(level_order)\n",
    "bars = axes[0, 0].bar(level_order, mean_spending.values, color=colors, alpha=0.8, edgecolor='black')\n",
    "axes[0, 0].set_title('各会员等级平均月消费金额', fontsize=12, fontweight='bold')\n",
    "axes[0, 0].set_ylabel('平均月消费(元)', fontsize=10)\n",
    "axes[0, 0].grid(axis='y', alpha=0.3)\n",
    "for bar, val in zip(bars, mean_spending.values):\n",
    "    height = bar.get_height()\n",
    "    axes[0, 0].text(bar.get_x() + bar.get_width()/2., height + 50,\n",
    "                    f'{val:.0f}', ha='center', va='bottom', fontsize=9)\n",
    "\n",
    "# 2. 各等级消费分布箱线图\n",
    "data_by_level = [df[df['会员等级']==level]['月消费金额'] for level in level_order]\n",
    "bp = axes[0, 1].boxplot(data_by_level, labels=level_order, patch_artist=True,\n",
    "                        boxprops=dict(facecolor='lightblue', alpha=0.7),\n",
    "                        medianprops=dict(color='red', linewidth=2))\n",
    "axes[0, 1].set_title('各会员等级月消费分布', fontsize=12, fontweight='bold')\n",
    "axes[0, 1].set_ylabel('月消费金额(元)', fontsize=10)\n",
    "axes[0, 1].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 3. 各等级客单价对比\n",
    "avg_price = df.groupby('会员等级')['客单价'].mean().reindex(level_order)\n",
    "axes[1, 0].barh(level_order, avg_price.values, color=colors, alpha=0.8, edgecolor='black')\n",
    "axes[1, 0].set_title('各会员等级平均客单价', fontsize=12, fontweight='bold')\n",
    "axes[1, 0].set_xlabel('平均客单价(元)', fontsize=10)\n",
    "axes[1, 0].grid(axis='x', alpha=0.3)\n",
    "for i, val in enumerate(avg_price.values):\n",
    "    axes[1, 0].text(val + 5, i, f'{val:.0f}', va='center', fontsize=9)\n",
    "\n",
    "# 4. 各等级用户数和总消费\n",
    "ax4_1 = axes[1, 1]\n",
    "ax4_2 = ax4_1.twinx()\n",
    "\n",
    "user_count = df['会员等级'].value_counts().reindex(level_order)\n",
    "total_spending = df.groupby('会员等级')['月消费金额'].sum().reindex(level_order)\n",
    "\n",
    "x_pos = range(len(level_order))\n",
    "ax4_1.bar(x_pos, user_count.values, alpha=0.6, color='steelblue', label='用户数')\n",
    "ax4_2.plot(x_pos, total_spending.values, color='red', marker='o', linewidth=2, markersize=8, label='总消费')\n",
    "\n",
    "ax4_1.set_xlabel('会员等级', fontsize=10)\n",
    "ax4_1.set_ylabel('用户数', fontsize=10, color='steelblue')\n",
    "ax4_2.set_ylabel('总消费金额(元)', fontsize=10, color='red')\n",
    "ax4_1.set_xticks(x_pos)\n",
    "ax4_1.set_xticklabels(level_order)\n",
    "ax4_1.set_title('各等级用户数与总消费', fontsize=12, fontweight='bold')\n",
    "ax4_1.legend(loc='upper left', fontsize=9)\n",
    "ax4_2.legend(loc='upper right', fontsize=9)\n",
    "ax4_1.grid(axis='y', alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n💡 会员等级洞察:\")\n",
    "print(f\"  1. 钻石会员平均消费({mean_spending['钻石']:.0f}元)是普通会员({mean_spending['普通']:.0f}元)的{mean_spending['钻石']/mean_spending['普通']:.1f}倍\")\n",
    "print(f\"  2. 虽然钻石会员人数最少,但单个用户价值极高\")\n",
    "print(f\"  3. 普通会员数量最多,是升级转化的重点群体\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第三阶段:用户分群分析(RFM模型)\n",
    "\n",
    "### 3.1 RFM模型构建\n",
    "\n",
    "RFM模型是经典的用户价值分析模型:\n",
    "- **R(Recency)**: 最近一次购买时间\n",
    "- **F(Frequency)**: 购买频率\n",
    "- **M(Monetary)**: 消费金额"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"=\"*100)\n",
    "print(\"👥 RFM用户分群分析\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 计算RFM指标(简化版,使用现有数据)\n",
    "# R: 用注册天数的倒数表示(注册越久,最近购买可能性越低)\n",
    "# F: 直接使用月购买次数\n",
    "# M: 直接使用月消费金额\n",
    "\n",
    "df_rfm = df[['用户ID', '注册天数', '月购买次数', '月消费金额']].copy()\n",
    "\n",
    "# 计算RFM分数(1-5分,5分最好)\n",
    "# R分数:注册天数越少越好\n",
    "df_rfm['R_Score'] = pd.qcut(df_rfm['注册天数'], q=5, labels=[5, 4, 3, 2, 1])\n",
    "# F分数:购买次数越多越好\n",
    "df_rfm['F_Score'] = pd.qcut(df_rfm['月购买次数'], q=5, labels=[1, 2, 3, 4, 5], duplicates='drop')\n",
    "# M分数:消费金额越高越好\n",
    "df_rfm['M_Score'] = pd.qcut(df_rfm['月消费金额'], q=5, labels=[1, 2, 3, 4, 5], duplicates='drop')\n",
    "\n",
    "# 转换为数值\n",
    "df_rfm['R_Score'] = df_rfm['R_Score'].astype(int)\n",
    "df_rfm['F_Score'] = df_rfm['F_Score'].astype(int)\n",
    "df_rfm['M_Score'] = df_rfm['M_Score'].astype(int)\n",
    "\n",
    "# 计算RFM总分\n",
    "df_rfm['RFM_Score'] = df_rfm['R_Score'] + df_rfm['F_Score'] + df_rfm['M_Score']\n",
    "\n",
    "# 用户分群\n",
    "def segment_user(row):\n",
    "    if row['RFM_Score'] >= 12:\n",
    "        return '重要价值客户'\n",
    "    elif row['RFM_Score'] >= 9:\n",
    "        if row['R_Score'] >= 4:\n",
    "            return '重要发展客户'\n",
    "        else:\n",
    "            return '重要保持客户'\n",
    "    elif row['RFM_Score'] >= 6:\n",
    "        if row['M_Score'] >= 3:\n",
    "            return '一般价值客户'\n",
    "        else:\n",
    "            return '一般发展客户'\n",
    "    else:\n",
    "        if row['R_Score'] <= 2:\n",
    "            return '流失预警客户'\n",
    "        else:\n",
    "            return '潜力客户'\n",
    "\n",
    "df_rfm['用户分群'] = df_rfm.apply(segment_user, axis=1)\n",
    "\n",
    "print(\"\\n【RFM分群结果】\")\n",
    "print(\"-\"*100)\n",
    "segment_stats = df_rfm['用户分群'].value_counts().sort_values(ascending=False)\n",
    "for segment, count in segment_stats.items():\n",
    "    print(f\"{segment:<15}: {count:>5}人 ({count/len(df_rfm)*100:>5.1f}%)\")\n",
    "\n",
    "# 合并回原数据\n",
    "df = df.merge(df_rfm[['用户ID', 'R_Score', 'F_Score', 'M_Score', 'RFM_Score', '用户分群']], on='用户ID')\n",
    "\n",
    "print(\"\\n【各分群平均指标】\")\n",
    "print(\"-\"*100)\n",
    "segment_metrics = df.groupby('用户分群').agg({\n",
    "    '月消费金额': 'mean',\n",
    "    '月购买次数': 'mean',\n",
    "    '客户评分': 'mean',\n",
    "    'R_Score': 'mean',\n",
    "    'F_Score': 'mean',\n",
    "    'M_Score': 'mean'\n",
    "}).round(2)\n",
    "print(segment_metrics.sort_values('月消费金额', ascending=False))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# RFM可视化\n",
    "fig, axes = plt.subplots(2, 2, figsize=(16, 12))\n",
    "\n",
    "# 1. 用户分群占比\n",
    "segment_sorted = segment_stats.sort_values(ascending=True)\n",
    "axes[0, 0].barh(segment_sorted.index, segment_sorted.values, color='steelblue', alpha=0.8, edgecolor='black')\n",
    "axes[0, 0].set_xlabel('用户数', fontsize=11)\n",
    "axes[0, 0].set_title('用户分群数量分布', fontsize=12, fontweight='bold')\n",
    "axes[0, 0].grid(axis='x', alpha=0.3)\n",
    "for i, val in enumerate(segment_sorted.values):\n",
    "    axes[0, 0].text(val + 10, i, f'{val} ({val/len(df)*100:.1f}%)', va='center', fontsize=9)\n",
    "\n",
    "# 2. RFM散点图(F vs M, 颜色表示R)\n",
    "scatter = axes[0, 1].scatter(df['F_Score'], df['M_Score'], c=df['R_Score'], \n",
    "                             cmap='RdYlGn', alpha=0.5, s=50, edgecolors='black', linewidth=0.5)\n",
    "axes[0, 1].set_xlabel('F Score (购买频次)', fontsize=11)\n",
    "axes[0, 1].set_ylabel('M Score (消费金额)', fontsize=11)\n",
    "axes[0, 1].set_title('RFM三维散点图', fontsize=12, fontweight='bold')\n",
    "cbar = plt.colorbar(scatter, ax=axes[0, 1])\n",
    "cbar.set_label('R Score (活跃度)', fontsize=9)\n",
    "axes[0, 1].grid(alpha=0.3)\n",
    "\n",
    "# 3. 各分群平均消费对比\n",
    "segment_spending = df.groupby('用户分群')['月消费金额'].mean().sort_values(ascending=False)\n",
    "colors_segment = ['red' if '重要' in x else 'orange' if '一般' in x else 'green' for x in segment_spending.index]\n",
    "bars = axes[1, 0].bar(range(len(segment_spending)), segment_spending.values, \n",
    "                      color=colors_segment, alpha=0.8, edgecolor='black')\n",
    "axes[1, 0].set_xticks(range(len(segment_spending)))\n",
    "axes[1, 0].set_xticklabels(segment_spending.index, rotation=45, ha='right', fontsize=9)\n",
    "axes[1, 0].set_ylabel('平均月消费(元)', fontsize=11)\n",
    "axes[1, 0].set_title('各分群平均月消费', fontsize=12, fontweight='bold')\n",
    "axes[1, 0].grid(axis='y', alpha=0.3)\n",
    "for bar, val in zip(bars, segment_spending.values):\n",
    "    height = bar.get_height()\n",
    "    axes[1, 0].text(bar.get_x() + bar.get_width()/2., height + 50,\n",
    "                    f'{val:.0f}', ha='center', va='bottom', fontsize=8)\n",
    "\n",
    "# 4. RFM得分分布\n",
    "rfm_score_dist = df['RFM_Score'].value_counts().sort_index()\n",
    "axes[1, 1].bar(rfm_score_dist.index, rfm_score_dist.values, color='coral', alpha=0.8, edgecolor='black')\n",
    "axes[1, 1].axvline(x=9, color='orange', linestyle='--', linewidth=2, label='高价值分界线(9分)')\n",
    "axes[1, 1].axvline(x=12, color='red', linestyle='--', linewidth=2, label='核心客户分界线(12分)')\n",
    "axes[1, 1].set_xlabel('RFM总分', fontsize=11)\n",
    "axes[1, 1].set_ylabel('用户数', fontsize=11)\n",
    "axes[1, 1].set_title('RFM得分分布', fontsize=12, fontweight='bold')\n",
    "axes[1, 1].legend(fontsize=9)\n",
    "axes[1, 1].grid(axis='y', alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n💡 RFM分群洞察:\")\n",
    "print(f\"  1. 重要价值客户占比{segment_stats.get('重要价值客户', 0)/len(df)*100:.1f}%,是核心用户群\")\n",
    "print(f\"  2. 流失预警客户占比{segment_stats.get('流失预警客户', 0)/len(df)*100:.1f}%,需重点挽留\")\n",
    "print(f\"  3. 潜力客户和一般发展客户是升级转化的重点目标\")\n",
    "print(f\"  4. 不同分群需要差异化的运营策略\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第四阶段:影响因素分析\n",
    "\n",
    "### 4.1 统计检验:不同群体的消费差异"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"=\"*100)\n",
    "print(\"📈 影响因素统计检验\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 1. 性别对消费的影响(t检验)\n",
    "print(\"\\n【1. 性别对月消费金额的影响 - 独立样本t检验】\")\n",
    "print(\"-\"*100)\n",
    "male_spending = df[df['性别']=='男']['月消费金额']\n",
    "female_spending = df[df['性别']=='女']['月消费金额']\n",
    "\n",
    "t_stat, p_value = stats.ttest_ind(male_spending, female_spending)\n",
    "print(f\"男性平均消费: {male_spending.mean():.2f}元\")\n",
    "print(f\"女性平均消费: {female_spending.mean():.2f}元\")\n",
    "print(f\"t统计量:      {t_stat:.4f}\")\n",
    "print(f\"p值:          {p_value:.6f}\")\n",
    "if p_value < 0.05:\n",
    "    print(\"✅ 结论: 性别对消费金额有显著影响(p<0.05)\")\n",
    "else:\n",
    "    print(\"❌ 结论: 性别对消费金额无显著影响(p≥0.05)\")\n",
    "\n",
    "# 2. 会员等级对消费的影响(ANOVA)\n",
    "print(\"\\n【2. 会员等级对月消费金额的影响 - 方差分析(ANOVA)】\")\n",
    "print(\"-\"*100)\n",
    "groups = [df[df['会员等级']==level]['月消费金额'] for level in level_order]\n",
    "f_stat, p_value = stats.f_oneway(*groups)\n",
    "\n",
    "print(f\"F统计量: {f_stat:.4f}\")\n",
    "print(f\"p值:     {p_value:.6f}\")\n",
    "if p_value < 0.05:\n",
    "    print(\"✅ 结论: 不同会员等级的消费金额存在显著差异(p<0.05)\")\n",
    "else:\n",
    "    print(\"❌ 结论: 不同会员等级的消费金额无显著差异(p≥0.05)\")\n",
    "\n",
    "# 3. 复购与消费的关系(t检验)\n",
    "print(\"\\n【3. 复购对月消费金额的影响 - 独立样本t检验】\")\n",
    "print(\"-\"*100)\n",
    "repurchase_yes = df[df['是否复购']=='是']['月消费金额']\n",
    "repurchase_no = df[df['是否复购']=='否']['月消费金额']\n",
    "\n",
    "t_stat, p_value = stats.ttest_ind(repurchase_yes, repurchase_no)\n",
    "print(f\"复购用户平均消费: {repurchase_yes.mean():.2f}元\")\n",
    "print(f\"非复购用户平均消费: {repurchase_no.mean():.2f}元\")\n",
    "print(f\"t统计量:          {t_stat:.4f}\")\n",
    "print(f\"p值:              {p_value:.6f}\")\n",
    "if p_value < 0.05:\n",
    "    print(\"✅ 结论: 复购用户与非复购用户的消费金额存在显著差异(p<0.05)\")\n",
    "    print(f\"   复购用户平均消费比非复购用户高{(repurchase_yes.mean()-repurchase_no.mean())/repurchase_no.mean()*100:.1f}%\")\n",
    "else:\n",
    "    print(\"❌ 结论: 复购用户与非复购用户的消费金额无显著差异(p≥0.05)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 回归分析:消费影响因素建模"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"🔮 多元线性回归:预测月消费金额\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "# 准备特征\n",
    "# 数值型特征\n",
    "numeric_features = ['年龄', '注册天数', '月登录次数', '月浏览时长', '月购买次数', '客户评分']\n",
    "\n",
    "# 分类变量编码\n",
    "df_encoded = df.copy()\n",
    "df_encoded['会员等级_编码'] = df_encoded['会员等级'].map(\n",
    "    {'普通': 1, '白银': 2, '黄金': 3, '铂金': 4, '钻石': 5}\n",
    ")\n",
    "df_encoded['城市等级_编码'] = df_encoded['城市等级'].map(\n",
    "    {'四线': 1, '三线': 2, '二线': 3, '一线': 4}\n",
    ")\n",
    "df_encoded['性别_编码'] = df_encoded['性别'].map({'男': 0, '女': 1})\n",
    "\n",
    "# 所有特征\n",
    "all_features = numeric_features + ['会员等级_编码', '城市等级_编码', '性别_编码']\n",
    "\n",
    "# 准备X和y\n",
    "X = df_encoded[all_features].values\n",
    "y = df_encoded['月消费金额'].values\n",
    "\n",
    "# 划分训练集和测试集\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
    "\n",
    "print(f\"\\n训练集大小: {X_train.shape[0]}个样本\")\n",
    "print(f\"测试集大小: {X_test.shape[0]}个样本\")\n",
    "print(f\"特征数量:   {X_train.shape[1]}个\")\n",
    "\n",
    "# 建立模型\n",
    "model = LinearRegression()\n",
    "model.fit(X_train, y_train)\n",
    "\n",
    "# 预测\n",
    "y_train_pred = model.predict(X_train)\n",
    "y_test_pred = model.predict(X_test)\n",
    "\n",
    "# 评估\n",
    "r2_train = r2_score(y_train, y_train_pred)\n",
    "r2_test = r2_score(y_test, y_test_pred)\n",
    "rmse_train = np.sqrt(mean_squared_error(y_train, y_train_pred))\n",
    "rmse_test = np.sqrt(mean_squared_error(y_test, y_test_pred))\n",
    "mae_test = mean_absolute_error(y_test, y_test_pred)\n",
    "\n",
    "print(\"\\n【模型评估】\")\n",
    "print(\"=\"*80)\n",
    "print(f\"{'指标':<20} {'训练集':<25} {'测试集':<25}\")\n",
    "print(\"=\"*80)\n",
    "print(f\"{'R²':<20} {r2_train:<25.4f} {r2_test:<25.4f}\")\n",
    "print(f\"{'RMSE(元)':<20} {rmse_train:<25.2f} {rmse_test:<25.2f}\")\n",
    "print(f\"{'MAE(元)':<20} {'-':<25} {mae_test:<25.2f}\")\n",
    "print(\"=\"*80)\n",
    "\n",
    "print(\"\\n【回归系数】\")\n",
    "print(\"=\"*80)\n",
    "print(f\"{'特征':<20} {'系数':<15} {'影响':<30}\")\n",
    "print(\"=\"*80)\n",
    "print(f\"{'截距':<20} {model.intercept_:<15.2f} {'-':<30}\")\n",
    "for feature, coef in zip(all_features, model.coef_):\n",
    "    impact = \"正向影响\" if coef > 0 else \"负向影响\"\n",
    "    print(f\"{feature:<20} {coef:<15.2f} {impact:<30}\")\n",
    "print(\"=\"*80)\n",
    "\n",
    "# 特征重要性(系数绝对值)\n",
    "feature_importance = pd.DataFrame({\n",
    "    '特征': all_features,\n",
    "    '系数': model.coef_,\n",
    "    '绝对值': np.abs(model.coef_)\n",
    "}).sort_values('绝对值', ascending=False)\n",
    "\n",
    "print(\"\\n【特征重要性排名】\")\n",
    "print(\"-\"*60)\n",
    "for i, row in feature_importance.head(5).iterrows():\n",
    "    print(f\"{feature_importance.index.get_loc(i)+1}. {row['特征']:<20}: 系数={row['系数']:>8.2f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 回归模型可视化\n",
    "fig, axes = plt.subplots(2, 2, figsize=(15, 12))\n",
    "\n",
    "# 1. 实际vs预测(测试集)\n",
    "axes[0, 0].scatter(y_test, y_test_pred, alpha=0.5, s=40, color='steelblue', edgecolors='black')\n",
    "min_val = min(y_test.min(), y_test_pred.min())\n",
    "max_val = max(y_test.max(), y_test_pred.max())\n",
    "axes[0, 0].plot([min_val, max_val], [min_val, max_val], 'r--', linewidth=2, label='理想预测线')\n",
    "axes[0, 0].set_xlabel('实际月消费(元)', fontsize=11)\n",
    "axes[0, 0].set_ylabel('预测月消费(元)', fontsize=11)\n",
    "axes[0, 0].set_title(f'实际 vs 预测 (测试集R²={r2_test:.4f})', fontsize=12, fontweight='bold')\n",
    "axes[0, 0].legend(fontsize=10)\n",
    "axes[0, 0].grid(alpha=0.3)\n",
    "\n",
    "# 2. 残差分布\n",
    "residuals = y_test - y_test_pred\n",
    "axes[0, 1].hist(residuals, bins=30, color='coral', alpha=0.7, edgecolor='black')\n",
    "axes[0, 1].axvline(x=0, color='red', linestyle='--', linewidth=2)\n",
    "axes[0, 1].set_xlabel('残差(元)', fontsize=11)\n",
    "axes[0, 1].set_ylabel('频数', fontsize=11)\n",
    "axes[0, 1].set_title('残差分布', fontsize=12, fontweight='bold')\n",
    "axes[0, 1].grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 3. 特征重要性\n",
    "top_features = feature_importance.head(8).sort_values('系数', ascending=True)\n",
    "colors = ['red' if x < 0 else 'green' for x in top_features['系数']]\n",
    "axes[1, 0].barh(top_features['特征'], top_features['系数'], color=colors, alpha=0.8, edgecolor='black')\n",
    "axes[1, 0].axvline(x=0, color='black', linestyle='-', linewidth=1)\n",
    "axes[1, 0].set_xlabel('回归系数', fontsize=11)\n",
    "axes[1, 0].set_title('Top8特征系数', fontsize=12, fontweight='bold')\n",
    "axes[1, 0].grid(axis='x', alpha=0.3)\n",
    "\n",
    "# 4. 预测误差分布\n",
    "error_pct = (residuals / y_test * 100)\n",
    "axes[1, 1].hist(error_pct, bins=30, color='lightgreen', alpha=0.7, edgecolor='black')\n",
    "axes[1, 1].axvline(x=0, color='red', linestyle='--', linewidth=2)\n",
    "axes[1, 1].set_xlabel('预测误差(%)', fontsize=11)\n",
    "axes[1, 1].set_ylabel('频数', fontsize=11)\n",
    "axes[1, 1].set_title('预测误差百分比分布', fontsize=12, fontweight='bold')\n",
    "axes[1, 1].grid(axis='y', alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n💡 模型洞察:\")\n",
    "print(f\"  1. 模型测试集R²={r2_test:.4f},能解释{r2_test*100:.1f}%的消费变化\")\n",
    "print(f\"  2. 测试集RMSE={rmse_test:.2f}元,平均预测误差约{rmse_test/y_test.mean()*100:.1f}%\")\n",
    "print(f\"  3. 影响消费的前3个因素: {', '.join(feature_importance.head(3)['特征'].values)}\")\n",
    "print(f\"  4. 模型可用于预测新用户的消费潜力\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第五阶段:业务洞察与策略建议\n",
    "\n",
    "### 5.1 关键发现总结"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"=\"*100)\n",
    "print(\"💡 数据分析关键发现\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "findings = {\n",
    "    '用户画像': [\n",
    "        f\"1. 用户年龄集中在{df['年龄'].quantile(0.25):.0f}-{df['年龄'].quantile(0.75):.0f}岁,平均{df['年龄'].mean():.1f}岁\",\n",
    "        f\"2. 性别分布相对均衡,女性占比{len(df[df['性别']=='女'])/len(df)*100:.1f}%\",\n",
    "        f\"3. 用户主要来自一二线城市({len(df[df['城市等级'].isin(['一线','二线'])])/len(df)*100:.1f}%)\",\n",
    "        f\"4. 普通会员占比最高({len(df[df['会员等级']=='普通'])/len(df)*100:.1f}%),钻石会员最稀缺({len(df[df['会员等级']=='钻石'])/len(df)*100:.1f}%)\"\n",
    "    ],\n",
    "    \n",
    "    '消费行为': [\n",
    "        f\"1. 月平均消费{df['月消费金额'].mean():.2f}元,中位数{df['月消费金额'].median():.2f}元(右偏分布)\",\n",
    "        f\"2. 平均每月购买{df['月购买次数'].mean():.1f}次,平均客单价{df['客单价'].mean():.2f}元\",\n",
    "        f\"3. 复购率为{len(df[df['是否复购']=='是'])/len(df)*100:.1f}%,复购用户平均消费比非复购用户高{(repurchase_yes.mean()-repurchase_no.mean())/repurchase_no.mean()*100:.1f}%\",\n",
    "        f\"4. 存在{len(df[df['是否异常消费']==1])}个异常高消费用户(月消费>{upper:.0f}元)\"\n",
    "    ],\n",
    "    \n",
    "    '用户分群': [\n",
    "        f\"1. 重要价值客户占比{segment_stats.get('重要价值客户', 0)/len(df)*100:.1f}%,平均消费{df[df['用户分群']=='重要价值客户']['月消费金额'].mean():.2f}元\",\n",
    "        f\"2. 流失预警客户占比{segment_stats.get('流失预警客户', 0)/len(df)*100:.1f}%,需重点挽留\",\n",
    "        f\"3. 潜力客户和一般发展客户合计占比{(segment_stats.get('潜力客户',0)+segment_stats.get('一般发展客户',0))/len(df)*100:.1f}%,是升级转化重点\",\n",
    "        f\"4. 不同用户群消费差异显著,需差异化运营\"\n",
    "    ],\n",
    "    \n",
    "    '影响因素': [\n",
    "        f\"1. 会员等级对消费影响最大,钻石会员消费是普通会员的{mean_spending['钻石']/mean_spending['普通']:.1f}倍\",\n",
    "        f\"2. 月购买次数与消费金额高度相关(r={df['月购买次数'].corr(df['月消费金额']):.3f})\",\n",
    "        f\"3. 客户评分与消费呈正相关(r={df['客户评分'].corr(df['月消费金额']):.3f}),满意度影响复购\",\n",
    "        f\"4. 预测模型R²={r2_test:.4f},可较准确预测用户消费潜力\"\n",
    "    ]\n",
    "}\n",
    "\n",
    "for category, items in findings.items():\n",
    "    print(f\"\\n【{category}】\")\n",
    "    print(\"-\"*100)\n",
    "    for item in items:\n",
    "        print(item)\n",
    "\n",
    "print(\"\\n\" + \"=\"*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2 运营策略建议"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"📋 差异化运营策略建议\")\n",
    "print(\"=\"*100)\n",
    "\n",
    "strategies = {\n",
    "    '重要价值客户': {\n",
    "        '特征': 'RFM得分≥12,高消费、高频次、高活跃',\n",
    "        '策略': [\n",
    "            '1. VIP专属服务:优先配送、专属客服、生日特权',\n",
    "            '2. 高端会员权益:积分加速、专享折扣、新品优先体验',\n",
    "            '3. 定期回馈:季度礼品、年度感恩回馈',\n",
    "            '4. 推荐奖励:老带新高额奖励'\n",
    "        ]\n",
    "    },\n",
    "    \n",
    "    '重要保持/发展客户': {\n",
    "        '特征': 'RFM得分9-11,有潜力成为核心用户',\n",
    "        '策略': [\n",
    "            '1. 会员升级引导:展示升级权益,设置升级任务',\n",
    "            '2. 精准营销:根据购买历史推荐相关商品',\n",
    "            '3. 限时优惠:专属折扣券,刺激消费频次',\n",
    "            '4. 满减活动:提升客单价'\n",
    "        ]\n",
    "    },\n",
    "    \n",
    "    '一般价值/发展客户': {\n",
    "        '特征': 'RFM得分6-8,中等消费水平',\n",
    "        '策略': [\n",
    "            '1. 品类拓展:推荐多品类商品,增加购买频次',\n",
    "            '2. 组合优惠:搭配套餐、满减活动',\n",
    "            '3. 签到奖励:每日签到领积分/优惠券',\n",
    "            '4. 社交裂变:分享得优惠,拼团活动'\n",
    "        ]\n",
    "    },\n",
    "    \n",
    "    '流失预警客户': {\n",
    "        '特征': 'R得分≤2,长时间未购买',\n",
    "        '策略': [\n",
    "            '1. 流失挽回:专属大额优惠券,限时使用',\n",
    "            '2. 问卷调研:了解流失原因,针对性改进',\n",
    "            '3. 新品推荐:根据历史偏好推送新品',\n",
    "            '4. 短信/推送提醒:温馨提示,唤醒沉睡用户'\n",
    "        ]\n",
    "    },\n",
    "    \n",
    "    '潜力客户': {\n",
    "        '特征': 'RFM得分<6,新用户或低活跃用户',\n",
    "        '策略': [\n",
    "            '1. 新手礼包:首单优惠、新人专享价',\n",
    "            '2. 会员权益教育:引导了解会员体系',\n",
    "            '3. 爆品推荐:高性价比商品,降低决策门槛',\n",
    "            '4. 限时秒杀:制造紧迫感,促成首单'\n",
    "        ]\n",
    "    }\n",
    "}\n",
    "\n",
    "for segment, info in strategies.items():\n",
    "    print(f\"\\n【{segment}】\")\n",
    "    print(\"-\"*100)\n",
    "    print(f\"用户特征: {info['特征']}\")\n",
    "    print(\"\\n运营策略:\")\n",
    "    for strategy in info['策略']:\n",
    "        print(f\"  {strategy}\")\n",
    "\n",
    "print(\"\\n\" + \"=\"*100)\n",
    "print(\"\\n【通用优化建议】\")\n",
    "print(\"-\"*100)\n",
    "print(\"1. 优化会员体系:明确各等级权益,设置合理的升级门槛\")\n",
    "print(\"2. 提升用户体验:关注客户评分低的用户,及时处理投诉\")\n",
    "print(\"3. 促进复购:针对非复购用户定向营销,提供复购优惠\")\n",
    "print(\"4. 数据驱动决策:建立用户消费预测模型,精准营销\")\n",
    "print(\"5. AB测试:不同策略进行小范围测试,优化营销效果\")\n",
    "print(\"=\"*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第六阶段:综合可视化报告"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 生成综合分析报告仪表盘\n",
    "fig = plt.figure(figsize=(20, 16))\n",
    "gs = fig.add_gridspec(4, 4, hspace=0.35, wspace=0.35)\n",
    "\n",
    "# 第一行:核心指标卡片(用文本模拟)\n",
    "kpi_data = [\n",
    "    ('总用户数', f\"{len(df):,}\", 'skyblue'),\n",
    "    ('平均月消费', f\"{df['月消费金额'].mean():.0f}元\", 'lightcoral'),\n",
    "    ('复购率', f\"{len(df[df['是否复购']=='是'])/len(df)*100:.1f}%\", 'lightgreen'),\n",
    "    ('平均评分', f\"{df['客户评分'].mean():.2f}分\", 'gold')\n",
    "]\n",
    "\n",
    "for idx, (label, value, color) in enumerate(kpi_data):\n",
    "    ax = fig.add_subplot(gs[0, idx])\n",
    "    ax.text(0.5, 0.6, value, ha='center', va='center', fontsize=28, fontweight='bold')\n",
    "    ax.text(0.5, 0.3, label, ha='center', va='center', fontsize=14)\n",
    "    ax.set_xlim(0, 1)\n",
    "    ax.set_ylim(0, 1)\n",
    "    ax.axis('off')\n",
    "    ax.add_patch(plt.Rectangle((0.05, 0.1), 0.9, 0.8, fill=True, color=color, alpha=0.3))\n",
    "\n",
    "# 第二行\n",
    "# 2-1: 会员等级分布\n",
    "ax1 = fig.add_subplot(gs[1, :2])\n",
    "level_counts = df['会员等级'].value_counts().reindex(level_order)\n",
    "ax1.bar(level_order, level_counts.values, color=colors, alpha=0.8, edgecolor='black')\n",
    "ax1.set_title('会员等级分布', fontsize=12, fontweight='bold')\n",
    "ax1.set_ylabel('用户数', fontsize=10)\n",
    "ax1.grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 2-2: 用户分群占比\n",
    "ax2 = fig.add_subplot(gs[1, 2:])\n",
    "segment_stats_plot = df['用户分群'].value_counts()\n",
    "ax2.pie(segment_stats_plot.values, labels=segment_stats_plot.index, autopct='%1.1f%%', startangle=90)\n",
    "ax2.set_title('用户分群占比', fontsize=12, fontweight='bold')\n",
    "\n",
    "# 第三行\n",
    "# 3-1: 消费金额分布\n",
    "ax3 = fig.add_subplot(gs[2, :2])\n",
    "ax3.hist(df['月消费金额'], bins=50, color='coral', alpha=0.7, edgecolor='black')\n",
    "ax3.axvline(df['月消费金额'].mean(), color='red', linestyle='--', linewidth=2, label=f\"均值={df['月消费金额'].mean():.0f}\")\n",
    "ax3.axvline(df['月消费金额'].median(), color='green', linestyle='--', linewidth=2, label=f\"中位数={df['月消费金额'].median():.0f}\")\n",
    "ax3.set_title('月消费金额分布', fontsize=12, fontweight='bold')\n",
    "ax3.set_xlabel('月消费金额(元)', fontsize=10)\n",
    "ax3.set_ylabel('用户数', fontsize=10)\n",
    "ax3.legend(fontsize=9)\n",
    "ax3.grid(axis='y', alpha=0.3)\n",
    "\n",
    "# 3-2: 各等级平均消费对比\n",
    "ax4 = fig.add_subplot(gs[2, 2:])\n",
    "mean_spending_plot = df.groupby('会员等级')['月消费金额'].mean().reindex(level_order)\n",
    "ax4.barh(level_order, mean_spending_plot.values, color=colors, alpha=0.8, edgecolor='black')\n",
    "ax4.set_title('各会员等级平均消费', fontsize=12, fontweight='bold')\n",
    "ax4.set_xlabel('平均月消费(元)', fontsize=10)\n",
    "ax4.grid(axis='x', alpha=0.3)\n",
    "\n",
    "# 第四行\n",
    "# 4-1: 相关系数TOP5\n",
    "ax5 = fig.add_subplot(gs[3, :2])\n",
    "top_corr_vars = corr_matrix['月消费金额'].drop('月消费金额').abs().sort_values(ascending=False).head(5)\n",
    "colors_corr = ['green' if corr_matrix['月消费金额'][v] > 0 else 'red' for v in top_corr_vars.index]\n",
    "ax5.barh(top_corr_vars.index, [corr_matrix['月消费金额'][v] for v in top_corr_vars.index], \n",
    "         color=colors_corr, alpha=0.8, edgecolor='black')\n",
    "ax5.axvline(x=0, color='black', linestyle='-', linewidth=1)\n",
    "ax5.set_title('与月消费相关性TOP5', fontsize=12, fontweight='bold')\n",
    "ax5.set_xlabel('相关系数', fontsize=10)\n",
    "ax5.grid(axis='x', alpha=0.3)\n",
    "\n",
    "# 4-2: 模型预测效果\n",
    "ax6 = fig.add_subplot(gs[3, 2:])\n",
    "ax6.scatter(y_test[:200], y_test_pred[:200], alpha=0.5, s=30, color='steelblue', edgecolors='black')\n",
    "min_v = min(y_test[:200].min(), y_test_pred[:200].min())\n",
    "max_v = max(y_test[:200].max(), y_test_pred[:200].max())\n",
    "ax6.plot([min_v, max_v], [min_v, max_v], 'r--', linewidth=2)\n",
    "ax6.set_title(f'预测效果(R²={r2_test:.3f})', fontsize=12, fontweight='bold')\n",
    "ax6.set_xlabel('实际消费(元)', fontsize=10)\n",
    "ax6.set_ylabel('预测消费(元)', fontsize=10)\n",
    "ax6.grid(alpha=0.3)\n",
    "\n",
    "plt.suptitle('电商平台用户运营分析综合报告', fontsize=18, fontweight='bold', y=0.995)\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n✅ 综合分析报告生成完成!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 课程总结\n",
    "\n",
    "### 完整数据分析流程回顾\n",
    "\n",
    "```\n",
    "1️⃣ 业务理解\n",
    "   - 明确分析目标\n",
    "   - 梳理业务问题\n",
    "   - 确定分析范围\n",
    "\n",
    "2️⃣ 数据准备\n",
    "   - 数据收集/生成\n",
    "   - 数据质量检查\n",
    "   - 数据清洗(缺失值、异常值)\n",
    "\n",
    "3️⃣ 探索性分析(EDA)\n",
    "   - 整体分布分析\n",
    "   - 相关性分析\n",
    "   - 分组对比分析\n",
    "\n",
    "4️⃣ 深度分析\n",
    "   - 用户分群(RFM模型)\n",
    "   - 统计检验(t检验、ANOVA)\n",
    "   - 预测建模(线性回归)\n",
    "\n",
    "5️⃣ 洞察提炼\n",
    "   - 关键发现总结\n",
    "   - 业务价值解读\n",
    "   - 策略建议输出\n",
    "\n",
    "6️⃣ 可视化呈现\n",
    "   - 综合报告仪表盘\n",
    "   - 多维度图表\n",
    "   - 清晰的结论展示\n",
    "```\n",
    "\n",
    "### 核心技能总结\n",
    "\n",
    "**数据处理**:\n",
    "- 数据清洗:缺失值填充、异常值检测\n",
    "- 数据转换:编码、标准化、特征工程\n",
    "- 数据整合:合并、分组、聚合\n",
    "\n",
    "**统计分析**:\n",
    "- 描述性统计:均值、中位数、标准差、分位数\n",
    "- 相关分析:相关系数矩阵、相关性检验\n",
    "- 假设检验:t检验、ANOVA、卡方检验\n",
    "\n",
    "**建模方法**:\n",
    "- 用户分群:RFM模型\n",
    "- 预测模型:线性回归\n",
    "- 模型评估:R²、RMSE、MAE\n",
    "\n",
    "**可视化技能**:\n",
    "- 基础图表:柱状图、饼图、散点图、箱线图\n",
    "- 高级图表:热力图、小提琴图、仪表盘\n",
    "- 图表美化:颜色、标签、网格、图例\n",
    "\n",
    "### 业务分析思维\n",
    "\n",
    "1. **问题驱动**:从业务问题出发,而非数据出发\n",
    "2. **假设验证**:提出假设,用数据验证\n",
    "3. **多维对比**:不同维度、不同群体的对比分析\n",
    "4. **因果推断**:区分相关性和因果性\n",
    "5. **可落地性**:分析结果要能转化为具体行动\n",
    "\n",
    "### 持续提升建议\n",
    "\n",
    "1. **多做项目**:实践是最好的老师\n",
    "2. **学习业务**:深入了解行业和业务知识\n",
    "3. **拓展技能**:机器学习、深度学习、大数据技术\n",
    "4. **沟通能力**:学会讲数据故事,说服决策者\n",
    "5. **工具熟练**:Pandas、SQL、BI工具、可视化工具"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 课后综合项目\n",
    "\n",
    "### 项目:在线教育平台学员分析\n",
    "\n",
    "某在线教育平台希望通过数据分析优化运营策略。数据包含:\n",
    "- 学员基本信息:年龄、学历、职业、所在城市\n",
    "- 学习行为:学习时长、完课率、测验成绩、登录频率\n",
    "- 付费信息:购买课程数、消费金额、是否续费\n",
    "- 满意度:评分、评价、投诉情况\n",
    "\n",
    "### 任务要求\n",
    "\n",
    "1. **数据准备**(20分)\n",
    "   - 生成至少1500条模拟数据\n",
    "   - 进行数据质量检查和清洗\n",
    "   - 处理缺失值和异常值\n",
    "\n",
    "2. **探索性分析**(30分)\n",
    "   - 整体数据分布分析(至少5个维度)\n",
    "   - 相关性分析\n",
    "   - 不同维度的对比分析(至少3个维度)\n",
    "\n",
    "3. **深度分析**(30分)\n",
    "   - 学员分群(可用RFM或K-Means)\n",
    "   - 影响因素分析(统计检验)\n",
    "   - 预测模型(预测学员消费或续费概率)\n",
    "\n",
    "4. **洞察与建议**(20分)\n",
    "   - 总结关键发现(至少10条)\n",
    "   - 提出运营策略建议(针对不同学员群体)\n",
    "   - 制作综合分析报告(可视化仪表盘)\n",
    "\n",
    "### 评分标准\n",
    "\n",
    "- 数据处理规范性(10分)\n",
    "- 分析方法的正确性(20分)\n",
    "- 分析的深度和广度(25分)\n",
    "- 可视化的清晰度和美观度(20分)\n",
    "- 洞察的深度和建议的可行性(25分)\n",
    "\n",
    "### 提交内容\n",
    "\n",
    "1. Jupyter Notebook文件(包含完整代码和分析过程)\n",
    "2. 分析报告PPT或Word(总结版)\n",
    "3. 数据文件(如果使用真实数据)\n",
    "\n",
    "---\n",
    "\n",
    "## 课程完结\n",
    "\n",
    "🎉 **恭喜你完成《商务数据采集与分析》第4阶段所有课程!**\n",
    "\n",
    "你已经掌握:\n",
    "- ✅ 完整的数据分析流程\n",
    "- ✅ Pandas数据处理核心技能\n",
    "- ✅ 统计分析方法\n",
    "- ✅ 机器学习基础(线性回归)\n",
    "- ✅ 数据可视化技巧\n",
    "- ✅ 业务分析思维\n",
    "\n",
    "**继续加油,成为优秀的数据分析师!** 💪\n",
    "\n",
    "---\n",
    "\n",
    "📧 如有疑问,请联系助教\n",
    "\n",
    "✅ 完成项目后,请提交完整作品"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
